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Introduction 

Probl  era 

The  Coast  Guard  (CG)  operates  42  healthcare  clinics 
throughout  the  United  States  and  Puerto  Rico.  The  policy  that 
directs  CG  clinics  to  evaluate  patient  satisfaction  lacks 
vision,  clear  constructs,  and  control  mechanisms.  As  a  result, 
each  clinic  administrator  is  responsible  to  design,  distribute, 
retrieve,  assess,  and  apply  patient  satisfaction  results 
individually. 

The  current  policy  provides  flexibility  of  processes,  as 
each  clinic  can  readily  pursue  analysis  addressing  local  care 
delivery  concerns.  However,  with  42  clinics  in  the  CG,  there  are 
42  different  ways  to  assess  patient  satisfaction.  With  few 
procedures  in  place  and  undefined  measurement  methods, 
variations  in  the  form  and  content  of  tools  that  evaluate 
patient  satisfaction  in  the  CG  clinics  are  evident. 
Unfortunately,  variation  of  surveys  leads  to  variations  in 
outputs,  impeding  the  benchmarking  or  comparison  of  scores.  In 
addition,  the  application  of  evidence-based  surveys  is  sporadic. 
Clinic  administrators  lack  the  technical  ability  and  resources 
to  design  and  implement  valid  measurement  tools,  thus  limiting 
quality  improvements  at  each  facility.  Furthermore,  scores  from 
a  wide  range  of  instruments  are  ill  suited  to  provide  systematic 
feedback  that  would  improve  health  care  system  as  a  whole.  In 
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the  absence  of  an  enterprise-wide  metric  for  patient 
satisfaction,  the  CG  lacks  the  ability  to  measure  and  monitor 
change  in  the  delivery  of  care  throughout  the  organization. 

This  study  consisted  of  the  defining  of  the  CG  processes, 
the  identification  of  desired  evaluation  dimensions,  and  the 
evaluation  of  two  alternative  policies.  The  evaluation  consisted 
of  a  cost  and  criteria  evaluation  which  culminated  in  a  Policy 
Analysis  Scorecard  (p.  65) .  Questions  answered  in  this  research 
include:  What  are  the  requirements  of  an  effective  policy  that 
support  the  evaluation  of  patient  satisfaction?  How  does  the  CG 
system  match  up  to  these  requirements?  And,  how  might  the  CG,  as 
an  organization,  improve  the  organizations  processes? 

The  extent  to  which  the  CG  policy  reflects  best  practices 
and  supporting  literature  was  substantiated  as  the  Status  Quo. 
These  findings  provide  a  baseline  for  current  CG  patient 
satisfaction  evaluation  processes.  Benchmarking  two  alternative 
policies  against  the  CG  baseline  provides  a  systematic  policy 
analysis  and  should  provide  a  context  for  developing  a  more 
robust  patient  feedback  process  within  the  CG. 

The  U.S.  Coast  Guard 

The  CG  is  the  smallest  branch  of  the  U.S.  armed  forces  and 
is  the  principal  federal  agency  for  providing  maritime  security. 
Its  41,000  active  duty  members  protect  95,000  miles  of  the  U.S. 
coast  line  (GAO,  2006;  Justice  &  Nimmich,  2006).  In  2002,  as  a 
result  of  the  disaster  at  the  World  Trade  Center  on  September 
11,  2001,  the  CG  command  was  reassigned  from  the  Department  of 
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Transportation  to  the  newly  formed  Department  of  Homeland 
Security  (DHS ) .  Despite  the  demands  of  the  Homeland  Security 
mission,  many  of  the  CG  traditional  missions  remained, 
including:  the  protection  of  marine  environments,  drug 
enforcement,  migrant  interdiction,  marine  safety,  waterway 
management,  defense  support,  and  search  and  rescue  operations. 
These  multi-mission  obligations  required  the  CG  organization  to 
adapt.  Since  2001,  the  CG  increased  its  active  duty  strength  by 
10%,  involuntarily  activated  over  4,000  reservists,  increased 
number  of  deployed  personnel,  and  increased  work  week  hours  (GAO, 
2006;  Justice  &  Nimmich,  2006;  USCG  Reservist  Magazine,  2003). 

All  of  these  initiatives  have  increased  the  demand  for  health 
care  by  CG  clinics. 

Delivery  of  Health  Care  in  the  CG 

The  CG  remains  the  only  branch  of  the  armed  services  not 
commanded  by  the  Department  of  Defense  (DOD) .  TRICARE  is  the 
Department  of  Defense's  worldwide  managed  care  program  for 
active  duty  and  retired  uniformed  service  members  and  their 
families,  and  survivors.  The  Military  Health  System  (MHS )  is  the 
military  component  of  the  TRICARE  health  management  system.  The 
MHS  consists  of  the  internal  delivery  of  care  provided  by  Army, 
Navy,  and  Air  Force  health  care  facilities  and  providers  whose 
primary  mission  is  to  ensure  the  nation  has  available  a  healthy 
fighting  force.  The  MHS  supports  combat  readiness  and  provides  a 
cost  effective,  quality  health  benefit  to  active  duty  members, 
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retirees,  survivors  and  their  families.  Because  the  CG  is  not 
part  of  the  DOD,  the  administration  of  CG  clinics  does  not  fall 
under  TRICARE,  so  many  administration  tools  present  in  the  MHS 
are  not  applied  to  the  CG  health  care  system. 

In  the  CG,  the  delivery  system  consists  of  42  ambulatory 
clinics  in  21  states  and  Puerto  Rico.  Primarily,  these  clinics 
provide  dental,  pharmacy,  and  primary  care  services,  and  can 
range  from  one  to  ten  providers.  Many  of  the  larger  clinics  offer 
specialty  services,  such  as  the  CG  Academy  Clinic,  which  provides 
mental  health  counseling.  In  addition,  the  smaller  clinics  are 
introducing  more  specialty  care  services  as  patient  needs  arise. 

Health  care  delivery  at  CG  clinics  is  primarily  focused  on 
the  readiness  of  CG  personnel  serving  on  active  duty.  Some  DOD 
personnel,  dependents,  retirees,  and  retiree  dependents  are 
treated  at  CG  clinics.  Due  to  limited  resources,  treating 
patients  other  than  active  duty  CG  members  occurs  only  when  space 
permits . 

To  optimize  delivery  as  well  as  accountability,  CG  personnel 
are  limited  in  their  selection  of  health  care  providers.  If 
location  is  not  a  factor,  CG  personnel  on  active  duty  must  first 
seek  care  in  a  CG  clinic.  The  scope  of  services  offered  at  CG 
clinics  is  limited.  Thus  CG  patients  seeking  specialty  care  must 
access  the  MHS  by  visiting  DOD  Military  Treatment  Facilities 
(MTFs ) ,  if  such  a  facility  exists  within  a  defined  radius  (i.e., 
an  hour's  drive) .  If  an  MTF  is  not  accessible,  the  CG  member  will 
receive  services  from  a  TRlCARE-approved  provider. 
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If  a  CG  clinic  is  not  available  for  primary  care,  and  CG 
personnel  are  within  an  hour's  drive  from  a  DOD  MTF,  the  CG 
member  must  enroll  as  a  DOD  MTF  beneficiary.  However,  many  CG 
personnel  serve  in  remote  areas  which  lack  CG  or  DOD  health  care 
services .  Personnel  are  then  assigned  to  TRICARE  network 
providers  in  the  civilian  sector.  Regular  and  routine  care  in 
rural  settings  may  be  limited,  but  is  usually  not  problematic. 
However,  CG  physicals  related  to  military  readiness  are  not 
supported  by  civilian  providers,  requiring  personnel  to  travel  to 
CG  clinics  or  DOD  MTFs  for  evaluation. 

Efficiency  of  care  in  the  CG  is  imperative.  Training,  as 
well  as  mission  obligations,  must  be  balanced  in  relation  to  care 
delivery.  CG  daily  operations  require  unit  deployments.  Remote  CG 
units  are  manned  efficiently,  with  each  person  assigned  to  the 
unit  deemed  essential.  If  one  member  of  the  team  is  unavailable 
due  to  medical  treatment  needs,  the  team  maybe  prevented  from 
deploying,  limiting  mission  capability.  Thus,  accessing 
immediate,  effective  health  care  services  is  essential. 

Supporting  the  rise  in  health  care  demand,  the  CG  has 
increased  the  number  of  its  health  care  services  adding  a  clinic 
in  St.  Petersburg,  FL  (2005)  and  a  clinic  in  San  Diego  (2006). 
Additionally,  the  CG  as  has  also  increased  its  association  with 
the  MHS ,  adding  CG  providers  to  DOD  MTF  facilities.  For  example, 
CG  clinic  personnel  in  Hawaii  were  reassigned  from  the  CG  base  to 
the  Tripler  Army  Medical  Center. 

According  to  CG  Headquarters  (LT  T.  Kulzer,  personal 
communication,  January,  2006),  providers  employed  at  CG  clinics 
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include  Public  Health  Service  (PHS)  personnel  (160+) ,  CG 
physician  assistants  (30+) ,  and  contract  providers  (60+) . 
Addressing  a  reported  shortage  of  over  30  PHS  providers  in  2006, 
and  in  response  to  the  increasing  need  for  specialty  services, 
the  use  of  contract  providers  has  increased  at  CG  clinics.  As  a 
result,  CG  clinic  administrators  report  service  quality  issues  in 
managing  care  due  to  limited  availability  of  PHS  providers, 
heavier  use  of  short-term  contract  providers,  and  excessive 
turnover . 

In  addition,  the  CG  responded  to  elevated  demand  on  its 
health  care  delivery  system  by  extending  the  capabilities  of  its 
volunteer  corps,  the  CG  Auxiliary.  CG  Auxiliary  personnel  have 
served  as  CG  providers  since  1999,  but  services  were  initially 
limited.  Because  readiness  requirements  increased  post  9-11,  CG 
Headquarters  authorized  the  broadening  of  Auxiliary  provider 
credentialing  as  a  means  to  assist  in  care  delivery  at  CG  clinics 
(U.S.  Coast  Guard,  2005). 

Concern  with  the  escalating  cost  of  services  associated  with 
medical  care  for  the  CG  has  also  led  to  increased  oversight. 
Lieutenant  Commander  (LCDR)  Turner,  in  a  report  presented  at  the 
MHS  Conference  (2007),  approximately  8%  of  the  2006  CG  budget  has 
been  allocated  for  the  provision  of  health  care.  He  estimates 
that  2%  of  the  budget  supports  internal  health  care  services, 
while  the  remaining  6%  supports  DOD/TRICARE  expenditures.  LCDR 
Turner  predicts  that  the  increasing  costs  of  CG  health  care  will 
outpace  other  CG  expenditures  over  the  next  10-15  years.  He  also 
predicts  that  as  more  funding  is  required  to  support  the  delivery 
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of  health  care  within  the  CG,  internal  requirements  for 
accountability  will  heighten.  His  recommendation  to  CG  health 
services  personnel  is  simply  to  become  more  efficient  at 
delivering  care. 

Evaluating  Patient  Satisfaction  in  the  CG 

Clinic  administrators  who  manage  CG  clinics  are  active  duty 
personnel  who  answer  directly  to  CG  commanders  responsible  for 
regional  front-line  operations.  Clinic  alignment  to  operational 
forces  reduces  the  influence  of  the  medical  chain-of -command 
within  the  organization,  allowing  each  CG  clinic  to  operate 
independently,  and  creating  diversity  in  structure,  process,  and 
outcomes.  Unfortunately,  the  competencies  required  to  be  a  CG 
clinic  administrator  are  ill-defined.  For  example,  there  is  no 
specific  or  required  skill  set.  A  senior  chief,  warrant  officer, 
lieutenant,  or  lieutenant  commander  could  serve  as  clinic 
administrator.  This  experiential  and  educational  diversity 
results  in  different  management  styles  and  variations  in  clinic 
operations.  More  specifically,  the  application  of  administrative 
tools,  such  as  the  evaluation  of  patient  satisfaction,  remains 
erratic . 

Within  the  CG,  the  evaluation  of  patient  satisfaction 
through  surveys  is  one  of  three  formal  ways  for  providing 
feedback.  Patient  advisory  committees  and  clinic  certification 
comprise  the  other  two.  Scores  from  patient  satisfaction  surveys 
are  submitted  to  local  patient  advisory  committees  for  review.  In 
addition,  the  process  of  evaluating  patient  satisfaction  for  each 
clinic  is  reviewed  during  accreditation  visits,  which  occur 
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approximately  every  three  years . 

In  2005,  the  CG  signed  an  agreement  with  the  Association  for 
Accreditation  of  Ambulatory  Health  Clinics  ( AAAHC )  to  accredit 
all  CG  clinics.  Accreditation  standards,  as  outlined  by  the 
AAAHC,  state  that  clinics  must  evaluate  patient  satisfaction. 
AAAHC  further  requires  survey  results  be  comparable,  although  the 
organization  neglects  to  define  comparable .  As  a  result 
interpretation  of  accreditation  standards  remains  problematic. 

Provisions  in  the  CG's  Medical  Manual  (2005)  require  clinics 
to  evaluate  patient  satisfaction.  Chapter  13. M  of  CG  Medical 
Manual  requires  clinics  to  survey  patient  satisfaction  annually. 
Surveys  must  also  be  available  for  patient  input  throughout  the 
year.  The  manual,  however,  does  not  specify  the  use  of  a 
particular  survey,  nor  does  it  dictate  content,  survey 
methodology,  data  analysis  or  application.  The  CG  medical  manual 
simply  states,  "A  local  form  is  authorized  for  use"  (p.  2). 

CG  medical  command  is  interested  in  whether  the  assessment 
of  patient  satisfaction  is  being  accomplished;  however,  specific 
levels  of  patient  satisfaction,  changes  in  patient  satisfaction, 
and  quality  improvements  generated  from  patient  satisfaction 
results  are  of  little  interest  (CG  Clinic  Administrators, 
personal  communication,  December  2006) .  Oversight  of  patient 
satisfaction  by  Maintenance  Logistics  Center  Pacific  Area 
(MLCPAC )  is  representative  of  this.  The  medical  chain  of  command 
at  MLCPAC  requires  notification  of  clinic  scores  only  when  there 
is  a  negative  score.  Defining  what  constitutes  a  negative  score 
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is  ambiguous,  leaving  clinic  administrators  to  determine  whether 
or  not  a  score  is  negative.  Consequently,  reporting  a  clinic's 
negative  score  may  indicate  adverse  performance.  The  process  is 
flawed  in  that  negative  reporting  serves  as  a  disincentive  to 
providing  straightforward  feedback.  Absence  of  oversight  further 
supports  an  under-reporting  environment. 

Because  clinics  operate  independently  and  the  process  for 
determining  patient  satisfaction  is  idiosyncratically  defined, 
each  clinic's  system  is  unique.  In  the  absence  of  common 
structure  and  process  controls,  assessing  patient  satisfaction  is 
problematic  at  best.  However,  the  new  commandant  has  called  for 
reorganization  of  CG  support  commands,  including  the  health  care 
system.  Currently  under  review  is  the  strengthening  of  the 
medical  chain  of  command  in  the  CG.  If  oversight  responsibilities 
of  the  CG  medical  command  are  strengthened,  an  opportunity  exists 
to  evaluate  the  patient  satisfaction  survey  process. 

Despite  the  differences  in  the  evaluation  of  patient 
satisfaction,  the  stakeholders  of  CG  health  care  policy  remain 
constant.  Defining  the  stakeholders  helps  bound  the  policy 
analysis.  A  stakeholder  analysis  is  an  important  visual  tool 
because  it  provides  an  organization  a  more  holistic  view  of 
actions  and  conditions  under  consideration  (Swayne,  Duncan,  & 
Ginter,  2006)  .  An  analysis  further  supports  the  evaluation  of  the 
different  proposed  policies  in  relation  to  current  environmental 


conditions . 
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This  analysis  was  conducted  using  a  policy  review  and 
interviews  with  CG  clinic  administrators  and  headquarter 
commands.  Questions  considered  during  the  analysis  of 
stakeholders  included:  Who  is  affected  by  patient  satisfaction  in 
CG  clinics?  Who  supports  the  process  of  assessing  patient 
satisfaction?  And  who  is  responsible  for  the  patient  satisfaction 
survey  process?  The  stakeholder  analysis  was  broken  down  into 
external  and  internal  components  to  identify  important  variables 
and  interrelationships  that  may  affect  decision-making  (Swayne, 
Duncan,  &  Ginter,  2006) . 

Identifying  internal  stakeholders  established  process 
responsibilities  and  communicated  the  diverse  needs  of  groups 
within  the  organization.  The  internal  analysis  revealed  eight  key 
stakeholders  (Figure  1) .  Six  of  the  stakeholders  have  an  active 
role  in  the  evaluation  and  application  of  the  scores.  The  other 
two  entities  CG  Headquarters  and  the  health  care  chain  of  command 
maintain  a  passive  role  in  survey  distribution  and  the 
application  of  results.  This  policy  analysis  evaluated 
strengthening  these  two  stakeholders  association  with  the 
implementation  of  a  standard  instrument  applied  throughout  the 
organization . 
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Figure  1.  Internal  stakeholder  analysis. 


Complementing  the  internal  stakeholder  analysis,  an  external 
stakeholder  analysis  is  included  (see  Figure  2) .  Because  some 
active  duty  service  members  other  than  CG  may  be  seen  at  CG 
facilities,  patients  are  included  in  both  internal  and  external 
stakeholder  analyses.  The  AAAHC ,  through  certification,  is  the 
only  other  external  stakeholder  that  may  interface  with  the  CG 
patient  satisfaction  process.  Subsequently,  the  stakeholder 
analysis  revealed  that  the  current  evaluation  process  limits 
direct  interaction  with  many  of  the  external  stakeholders.  A  more 
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formalized  evaluation  system  provides  additional  opportunities 
for  improved  communication  with  external  stakeholders. 

Figure  2.  External  stakeholder  analysis. 


Evidence 

In  the  2001  landmark  report  Crossing  the  Quality  Chasm:  A 
New  Health  System  for  the  21st  Century,  the  Institute  of  Medicine 
( IOM)  identified  the  need  for  fundamental  changes  in  the 
organization  and  delivery  of  health  care  in  the  United  States. 
Furthermore,  the  IOM  asserted  that  patient  satisfaction  surveys 
that  accurately  assessed  patient  care  could  help  improve 
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quality.  The  IOM  endorsed  Lohr's  definition  of  quality :  "The 
degree  to  which  health  services  for  individuals  and  populations 
increase  the  likelihood  of  desired  health  outcomes  and  are 
consistent  with  current  professional  knowledge"  (p.  232) . 

Despite  the  IOM's  call  for  fundamental  change  that 
emphasized  patient-centered  care  and  patient  satisfaction,  the 
CG's  evaluation  of  patient  satisfaction  remained  stagnant.  In 
addition,  post  9-11  requirements  dictated  internal  changes  to 
care  delivery  devoid  of  metrics  to  evaluate  the  effects  on 
patients.  In  the  commercial  world  of  health  care  delivery, 

patient  satisfaction  is  valued.  Patients  are  consumers;  those 
dissatisfied  with  care  may  receive  treatment  elsewhere,  which 
potentially  affects  the  financial  well-being  of  particular 
practices  (Press,  2006). 

Influencing  patient  choice  is  not  the  only  reason  for 
evaluating  patient  satisfaction.  Orlando  and  Meredith  (2002) 
report  high  patient  satisfaction  is  significantly  and  positively 
related  to  high  quality  care.  Patients  with  higher  satisfaction 
express  more  confidence  and  trust  in  their  providers.  Trust 
results  in  greater  willingness  to  disclose  symptoms,  which 
increases  the  doctors'  opportunities  to  provide  effective  care 
(Orlando  &  Meredith) .  Other  researchers  also  report  a  significant 
correlation  between  patient  satisfaction  and  high-quality 
outcomes  and  the  delivery  of  efficient  health  care  (Alazri  & 

Neal,  2003;  IOM,  2001;  Ransom,  Joshi,  &  Nash,  2005). 

Additionally,  Alazri  and  Neal  (2003)  report  a  significant 
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positive  association  between  patient  satisfaction,  improved  staff 
culture,  and  cost  savings. 

Patient  satisfaction  surveys  provide  a  patient-based  measure 
that  addresses  non- technical  aspects  of  health  care  (IOM,  2001; 
Press,  2006).  Furthermore,  Drain  and  Clark  (2004)  assert 
surveying  patient  satisfaction  is  good  practice  because,  "A 
patient  centered  organization  is  committed  to  affirming  patients' 
perceptions  as  their  reality  and  improving  the  way  patients 
experience  care  in  the  future"  (p.  2). 

Research  in  the  Military 

The  DOD  has  completed  several  patient  satisfaction  studies; 
however,  none  have  evaluated  care  at  CG  clinics.  Yet,  many 
findings  may  be  applied  in  CG  settings.  Concurring  with  civilian 
research,  DOD  findings  support  the  notion  that  patient 
satisfaction  is  an  indicator  of  health  care  quality  (Jackson  & 
Kroenke,  1997;  Manglesdorff  &  Finstuen,  2003;  Manglesdorf f , 
Finstuen,  Larsen,  &  Weinberg,  2005) .  Additional  studies  have 
validated  the  use  of  patient  satisfaction  results  because  high 
ratings  of  patient  satisfaction  in  military  facilities  correlate 
with  increased  military  readiness  and  regimen  compliance, 
continuation  of  care,  and  lower  health  care  costs  (Alazri  &  Neal, 
2003;  Fan,  Burman,  McDonnell  &  Fihn,  2005;  Hulka,  Cassel,  Kupper, 
&  Burdette  1976) . 

As  managed  care  became  the  predominant  MHS  model,  the  1993 
National  Defense  Authorization  Act  (NDAA)  mandated  that  the 
armed  services  establish  a  formal  evaluation  process  to  assess 
patient  satisfaction.  In  response,  the  DOD  established  several 
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different  surveys.  The  principal  means  for  providing  health  care 
was  increasing  influenced  by  managed  care  throughout  the  1990s. 

As  a  result,  new  concerns  arose  regarding  the  adequacy  of  care, 
and  in  1998,  Congress  mandated  a  review  of  collecting  patient 
feedback.  The  Government  Accountability  Office  (GAO)  conducted  a 
study  entitled,  "Defense  Health  Care:  DOD  Could  Improve  Its 
Beneficiary  Feedback  Approaches".  The  report  cited  that  a  lack  of 
standards  for  evaluations  prevented  systematic  reporting,  as  each 
MTF  assessed  had  "differing  ways  of  obtaining,  documenting,  and 
analyzing  beneficiary-imitated  feedback  systems"  (GAO,  1998, 
p.13).  The  findings  of  this  report  pushed  the  DOD  to  standardize 
the  evaluation  of  surveys,  which  supported  the  transition  of  the 
MHS  to  be  a  more  outcomes-oriented  system  of  care. 

Standard  Survey  Best  Practices 

One  standard  survey  used  by  the  DOD  is  the  Health  Care 
Survey  of  DOD  Beneficiaries  (HCSDB) .  This  survey  is  based  on  the 
original  Consumer  Assessment  of  Health  Plans  Survey  (CAHPS) .  The 
CAHPS  program  was  developed  by  the  Agency  for  Healthcare 
Research  and  Quality  (AHRQ)  in  conjunction  with  the  Centers  for 
Medicare  and  Medicaid  Services  and  the  U.S.  Department  of  Health 
and  Human  Services  "to  develop  a  national  standard  for  assessing 
patient  experiences  of  hospital  care  in  the  United  States" 

(Drain  &  Clark,  2004,  p.  1) .  Because  CAHPS  surveys  are  widely 
used,  the  results  allow  for  benchmark  comparisons  with  scores 
from  civilian  care  facilities.  The  Medical  Management  Guide 
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(2006)  published  by  DOD  supports  the  comparing  of  scores  and 
defines  benchmarking  as  "the  continuous  process  of  measuring 
process,  services,  and  practices  against  industry  standards  to 
compare  performance  and  gauge  where  and  whether  efforts  to 
improve  might  be  indicated"  (p.  VI. 8).  The  results  of  HCSDB 
surveys  encourage  MTFs  to  analyze  trends  quarterly  and  allows 
for  benchmarking  among  DOD  and  commercial  facilities. 

The  HCSDB  survey  requires  respondents  to  answer  90+ 
questions  consisting  of  core  content  that  changes  little  from 
year  to  year,  and  includes  supplementary  material  that  is 
updated  quarterly.  Supplementary  questions  allow  the  analysis  of 
specific  concerns;  for  example,  responses  may  include  feedback 
regarding  patients'  perceptions  of  health  care  delivery 
following  base  realignment  and  closure.  This  query  is  pertinent 
for  assessing  perceptions  related  to  time-specific  access. 

The  HCSDB  results  are  available  on  TRICARE's  Website 
(www.tricare.osd.mil)  and  include  ratings  of  health  care  plans, 
personal  provider  ratings,  and  specialist  ratings.  Results  are 
aggregated  by  DOD  service  categories  so  that  the  Army,  Air 
Force,  and  Navy  may  identify  data  trends.  Data,  however,  are  not 
aggregated  for  the  CG.  According  Dr.  Williams,  Director,  Center 
for  Health  Care  Management  Studies,  TRICARE  Management  Office 
(personal  communication,  April  2007),  analysis  of  CG  data  is  not 
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included  because  of  the  relatively  few  numbers  of  CG  patients 
sampled. 

Similar  to  the  DOD,  the  VA  has  also  established  standard 
surveys  to  evaluate  organizational  care  delivery.  However,  the 
evaluation  of  patient  satisfaction  in  the  VA  in  the  early  1990s 
lacked  cohesion.  Like  the  CG  of  today,  the  VA  permitted  each 
local  facility  to  monitor  patient  satisfaction  with  minimal 
oversight  (Humble,  Schafer,  &  Fleming,  2004)  .  According  to 
Humble,  Schafer,  and  Fleming  (2004) , 

Though  these  local  surveys  may  have  been  adequate  for 
quality  improvement  purposes,  they  were  usually  side-duty 
of  staff  inexperienced  in  survey  methods  and  statistical 
analysis.  Without  a  standard  system-wide  process,  it  wasn't 
possible  to  develop  a  national  data  and  compare  facility 
scores  across  the  system,  (p.  1) 

Transition  to  standard  surveys  for  the  organization  came  in  1995 
when  the  VA  conducted  its  first  nationwide  patient  satisfaction 
survey.  Soon,  the  assessment  by  individual  facilities  became 
obsolete,  and  standardized  tools  with  strategic  intent  were 
adopted. 

In  2002,  the  VA  began  using  the  Survey  of  Healthcare 
Experiences  of  Patients  [SHEP]  (VA  Form  10-1454-3)  throughout 
the  organization  as  a  gauge  for  measuring  patient  satisfaction 
and  clinic  expectations.  SHEP  was  developed  by  using  focus 
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groups  and  Picker  Institute  research.  SHEP  serves  both  as  a 
means  for  quality  improvement  and  as  a  performance  measure 
(Humble,  Schaefer,  &  Fleming,  2004).  According  to  the  VA  (2007), 
the  SHEP  requires  approximately  30  minutes  to  complete  and 
consists  of  107  items.  The  SHEP  survey  is  distributed  via  mail 
to  patients  of  the  800  outpatient  VA  clinics.  Results  are  posted 
on  the  VA  Website. 

SHEP  addresses  the  following  standards  of  health  (Humble, 
Schaefer,  &  Fleming,  2004,  p.  4): 

•  Provide  timely  access  to  health  care 

•  Treat  patients  with  courtesy  and  respect 

•  Support  patient's  emotional  needs 

•  Provide  information  and  education  about  condition, 
treatments  and  tests 

•  Have  one  provider  or  team  in  charge  of  care 

•  Coordinate  both  visit  specific  and  overall  health  care 
needs 

•  Insure  patient  involvement  in  decisions  about  care 

•  Meet  physical  comfort  needs 

•  Provide  timely  and  appropriate  pharmacy  services 

•  Provide  timely  and  appropriate  pharmacy  services 

•  Provide  a  smooth  transition  between  inpatient  and 


outpatient  care 
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•  Rate  overall  VA  health  care 

Questions  on  the  SHEP  are  organized  into  subsections,  allowing 
for  feedback  regarding  several  different  dimensions.  These 
sections  categorize  feedback  regarding  recent  visits,  scheduling 
appointments,  registration,  and  efficacy  of  care,  and  an  overall 
impression  of  the  most  recent  visit.  The  survey  also  requests 
feedback  for  clinic  visits  during  the  previous  two  months, 
including  specialist  clinics,  pharmacy  use,  and  an  overall 
impression  of  the  VA  clinic  system.  SHEP  concludes  with  38  items 
related  to  demographics  and  patient  health. 

Responses  to  items  in  these  subsections  permit  analyzing 
perceptions  of  care  leading  to  specific  initiatives  to  improve 
care  quality.  Despite  this  tool's  potential  to  support 
continuous  improvements  in  care,  its  length  markedly  exceeds  the 
scope  of  items  on  surveys  used  at  CG  clinics.  The  number  of 
items  may  thwart  efforts  to  implement  a  similar  survey  within 
the  CG's  system.  Clinic  administrators  (personal  communication, 
December  2007),  conceded  that  the  length  of  patient  satisfaction 
questionnaires  is  limited  by  CG  operational  commanders,  as  there 
is  a  perception  of  survey  fatigue.  In  addition,  since  the  CG 
clinic  administrators  conduct  analyses  by  hand,  a  107-question 
survey  would  not  be  supported  by  current  resource  allocation. 

Complementing  the  lengthy  SHEP  survey  is  the  VA's  "quick 
card"  approach.  The  survey  may  be  completed  during  a  clinic 
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visit  or  on-line.  Results  may  be  used  for  immediate  quality 
improvement.  The  VA  estimates  that  patients  can  complete  the 
survey  in  approximately  five  minutes.  The  "quick  card"  form 
contains  eight  questions  that  query  the  courtesy  of  the  staff, 
the  timeliness  of  service,  confidence  in  the  provider,  respect 
for  patient  privacy,  staff  sensitivity  to  patient  concerns, 
cleanliness  of  the  facility,  health  information,  and  overall 
patient  satisfaction  (Department  of  Veterans  Affairs,  n.d.). 

This  survey  also  includes  a  section  for  comments  and 
suggestions,  but  no  demographic  data  are  collected. 

Consequently,  many  current  MHS  and  VA  initiatives  have  been 
linked  mandates  by  Congress.  Often  cited  is  the  1993  National 
Defense  Authorization  Act,  which  requires  that  the  armed 
services  adopt  a  formal  system  to  assess  patient  satisfaction. 
Unfortunately,  the  meaning  of  a  formal  evaluation  has  never  been 
defined.  Since  then  however,  MHS  and  the  VA  have  adopted  system- 
wide  tools.  In  the  MHS  and  VA  scores  of  these  system  wide  tools 
have  been  linked  to  strategic  intent,  financial  incentives, 
quality  awards,  and  whose  results  are  benchmarked  among 
facilities  (Fortney,  Borowsky,  &  Chapko,  1999;  MHS  Medical 
Manual,  2006) . 

Facilities  external  to  military  care  have  also  transitioned 
their  patient  satisfaction  policies  to  standard  patient 
evaluations  within  their  organizations;  two  such  entities  are  the 
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National  Committee  for  Quality  Assurance  and  the  Agency  for 
Healthcare  Research  and  Quality.  Policies  of  both  organizations 
support  the  marketing  and  benchmarking  of  findings 
organizational-wide  surveys.  Even  though  the  transparency  of 
scores  has  not  yet  been  associated  with  affecting  consumer 
choice,  Hibbard,  Stockard,  and  Tusler  (2003)  assert  the 
transparency  of  scores  has  proven  beneficial,  stating  that  a 
desire  to  enhance  reputations  drives  improvement  efforts. 
Researchers  also  provide  strong  evidence  that  the  marketing  of 
quality  measurements  stimulates  quality  improvement  activities, 
thus  facilities  that  withhold  results  limit  quality  improvement 
efforts  (Hibbard,  Stockard,  &  Tusler) . 

Policy  Options 

The  purpose  of  this  study  was  to  conduct  a  policy 
assessment  of  the  evaluation  of  patient  satisfaction  within  the 
CG.  This  analysis  determined  what  an  effective  evaluation 
process  should  consist  of.  In  addition,  the  comparative  results 
of  this  analysis  supports  a  policy  shift  for  the  CG.  Expanding 
the  scope  of  evaluating  patient  satisfaction  through  the 
standardization  of  metrics  by  employing  a  visit-specific  survey 
uniformly  through  out  the  organization  is  recommended.  A 
standard  survey  that  assists  local  clinic  operations  in 
identifying  quality  improvements,  but  also  supports  strategic 
management  of  the  CG  health  care  system  as  a  whole  should  be  the 


desired  end  state. 
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Specifically,  the  current  policy  (Option  1)  was  evaluated 
against  a  Web-based  survey  designed  in-house  by  the  CG  (Option 
2)  and  a  standard  survey  and  analysis  process  established  by  the 
American  Medical  Group  Association  (Option  3) .  Policy  Options  2 
and  3  proposed  the  implementation  of  a  standard  survey  capable 
of  evaluating  patient  care  uniformly.  Ratings  of  patient 

satisfaction  collected  from  a  standard  patient  survey  and  process 
encourage  benchmarking  among  CG  clinics.  Bogan  &  English  (1994) 
define  benchmarking  as  the  "systematic  process  of  searching  for 
best  practices,  innovative  ideas,  and  highly  effective  operating 
procedures  that  lead  to  superior  performance"  (p.  1) .  Results 
assessed  from  a  standard  survey  may  also  support  an 
organizational  metric  that  could  be  linked  to  strategic 
management  efforts. 
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Option  1  -  Status  Quo 

Option  1,  Status  Quo,  reflects  the  current  CG  policy. 
Present  CG  policy  empowers  clinic  administrators  to  assess 
patient  satisfaction  with  minimal  oversight  from  the 
organization.  A  review  of  13  of  the  42  clinic  surveys  and 
interviews  with  the  clinic  administrators  were  conducted  to 
establish  a  baseline  of  patient  satisfaction  practices  within 
the  CG.  Clinic  surveys  evaluated  were  visit-specific  surveys, 
thus  questions  directed  patients  to  evaluate  care  of  the  most 
recent  visit. 

All  clinics  reported  internal  survey  administration.  The 
preponderance  of  clinics,  11  in  all,  query  patient  satisfaction 
by  administering  a  paper-and-pencil  survey;  telephone  or  mailed 
surveys  were  not  used.  In  addition  to  the  paper-and-pencil 
survey,  2  of  the  11  clinics  also  provided  an  electronic  survey 
via  the  Internet  to  receive  patient  feedback  throughout  the 
year . 

The  two  remaining  clinics  use  electronic  surveys,  wherein 
patients  received  a  survey  via  e-mail  following  their  visits. 
This  survey  delivery  method,  used  in  both  locations,  was 
initiated  by  a  clinic  administrator  using  Dynamic  Surveys,  a 
software  program  available  to  all  CG  units.  Dynamic  Surveys 
supports  an  automatic  analysis  of  feedback,  thus  enabling  the 
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administrator  to  conduct  surveys  monthly,  a  frequency  that 
exceeds  the  CG  standard  and  the  norm. 

Option  2  -  CG  Designed  Survey  Web  Interface 

The  next  option  assessed,  Option  2,  CG  Designed  Survey  Web 
Interface,  considered  a  survey  instrument  designed  internally  by 
the  CG  to  address  organizational  needs.  Specifically,  a  CG 
developed  survey  would  evaluate  medical,  dental,  and  pharmacy 
services  concurrently.  An  in-house  designed  survey  must  be 
developed  by  a  project  manager  under  the  guidance  of  medical 
command  headquarters .  A  senior  headquarters  representative  would 
be  responsible  for  allocating  resources  and  providing 
authorization  for  proposals  submitted.  The  project  manager  would 
coordinate  an  issue-specific  workshop  which  would  involve  8-12 
subject  matter  experts  meeting  for  a  three-day  conference.  The 
purpose  of  the  conference  would  be  to  establish  vision,  identify 
survey  content  and  form,  develop  an  implementation  plan,  and 
construct  policy.  Attendees  would  review  current  CG  norms  and 
best  practices  in  order  to  develop  a  path  forward. 

The  uniform  survey  to  be  developed  is  expected  to  be 
reflective  of  current  practices  in  the  CG,  and  specific  to  CG 
health  care  delivery  needs.  The  expected  uniform  survey  would 
consist  of  a  Web-based  interface  and  would  be  limited  to  15-25 
category  response  questions,  including  demographics,  and 
complemented  by  an  open-ended  comments  section. 
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The  inclusion  of  sections  that  allow  verbatim  comments  is 
validated  by  Press's  (2006)  description,  "Comments  written  by 
patients  are  essentially  the  simplest,  most  direct  and 
universally  comprehensible  pieces  of  data  on  the  survey.  Comments 
can  be  invaluable"  (p.  101).  Although  verbatim  questions  require 
content  analysis,  their  inclusion  constitutes  an  important  survey 
element,  as  open-ended  responses  add  qualitative  element  to 
quantitative  scoring  methods  (White,  1999) . 

Survey  distribution  to  be  developed  would  be  accomplished 
through  the  use  of  e-mail,  as  facilitated  by  Design  Surveys. 

Using  the  software  system  to  distribute  a  uniform  survey  is 
beneficial,  as  it  reduces  data  entry  and  internal  analyses,  and 
supports  benchmarking  of  scores  among  CG  clinics.  Currently,  two 
CG  clinics  employ  Design  Surveys  to  evaluate  patient 
satisfaction,  as  stated  previously. 

Pilot  testing  establishes  survey  validity  and  reliability, 
though  this  testing  is  expected  to  be  minimal.  Due  to  limited 
resources  within  the  CG,  it  is  not  standard  CG  policy  to  test 
in-house  management  tools  against  established  scientific 
standards . 

The  flexibility  of  having  the  CG  determine  the  content  of 
the  survey  and  its  format  is  a  plus.  Design  flexibility  comes  at 
a  cost;  the  development  and  implementation  of  Option  2  is 
projected  to  cost  substantially  more  than  the  other  options. 
However,  Web-based  distribution  that  supports  automatic  analyses 
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is  expected  to  constitute  the  lowest  annual  survey  costs  of  all 
three  options . 

Option  3  -  AMGA  Survey 

Option  3,  the  American  Medical  Group  Association  (AMGA) 
Survey,  proposes  implementation  of  the  AMGA  patient  satisfaction 
survey.  The  AMGA  uses  VSQ-9,  an  evidence-based  survey  tool 
created  by  the  RAND  Corporation  as  the  basis  for  its  Patient 
Satisfaction  Benchmarking  Program.  The  AMGA  survey  form  does  not 
permit  unsolicited  comments,  but  AMGA  encourages  clinic  staff 
members  to  instruct  respondents  to  write  comments  on  the  back  of 
the  surveys  (Appendix  A) .  Since  verbatim  responses  are  used  in 
Option  1,  Status  Quo,  and  projected  to  be  incorporated  in  Option 
2,  CG  Design  Survey  Web  Interface,  the  evaluation  of  Option  3 
may  include  the  evaluation  of  written  responses. 

Surveys  are  distributed  to  patients  at  the  clinic  by  staff 
members .  Respondents  return  the  surveys  through  drop  boxes ,  a 
common  practice  in  many  CG  clinics.  Surveys  are  then  collected 
and  sent  to  AMGA,  which  scans  responses  to  create  a  database  for 
analysis . 

Surveys  are  analyzed  during  the  first  part  of  the  month  and 
reports  are  returned  to  a  medical  facility  within  30  days. 
According  to  information  on  the  AMGA  website  (www.amga.org),  the 
survey  program  analyzes  the  performance  of  each  clinic  site  as 
well  as  individual  providers.  The  program  also  provides 
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graphical  comparisons  to  regions  and  specialty-specific  norms. 
This  information  should  help  identify  "best  practice" 
benchmarks,  and  increase  opportunities  to  target  quality 
improvements.  The  AMGA  benchmarking  program  offers  eight  survey 
periods  throughout  the  year;  however,  the  CG  is  expected  to 
evaluate  clinics  only  twice  per  year. 

Evaluative  Criteria 

A  scorecard  that  evaluated  the  strengths  and  weaknesses  of 
the  three  policy  options  was  created.  This  approach,  where 
strengths  and  weaknesses  were  determined,  promoted  comparisons, 
and  enabled  the  consideration  of  impacts  that  cannot  be  expressed 
in  quantitative  terms  (Ruchelman,  1984) .  Implementation  costs, 
operating  costs,  and  opportunity  costs  were  determined  and 
presented  in  the  scorecard.  Also,  the  different  options  were 
evaluated  against  set  criteria  to  determine  the  best  solution  for 
the  CG. 

Criteria  was  established  to  evaluate  the  policies.  Selecting 
criteria  was  subjective,  but  was  based  on  research  and 
benchmarking  of  external  policies.  Drain  and  Clark  (2006)  note 
that  surveys  are  processes  that  combine  steps,  both  iterative  and 
interdependent;  i.e.,  decisions  made  at  one  point  of  the  survey 
design  affect  subsequent  steps  of  the  survey  process.  This  point 
substantiates  the  assertion  that  a  policy  must  direct  all  steps 
in  the  process.  Because  of  this  interdependency  between  steps, 
outcomes  must  be  evaluated  throughout  the  policy  analysis  in 
order  to  determine  the  best  option. 
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This  study  used  the  survey  process,  which  is  conceptually 
defined  in  five  phases:  design,  distribution,  retrieval, 
analysis,  and  application  of  results,  as  the  foundation  for  the 
criteria  analysis.  This  high  level  process  was  derived  from  Aday 
and  Cornelius  text  Designing  and  Conducting  Health  Surveys 
(2006).  The  five  high-level  phases  presented  in  Figure  1.  depicts 
visibly  the  inside  workings  of  a  process,  captures  types  of 
decisions  being  made,  and  identifies  possible  measurement  points 
(Montgomery,  2002)  .  Aligned  with  the  five  phases  of  the  process, 
the  criteria  which  was  evaluated  is  presented.  Reasoning  for 
selecting  the  desired  dimensions  is  supported  in  the  following 
text . 
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Figure  3.  Patient  satisfaction  survey  process  flowchart. 
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Figure  3.  This  conceptual  flowchart  for  patient  satisfaction 
survey  process  is  presented  in  relation  to  dimensions  evaluated 
in  the  policy  review.  The  flowchart  is  derived  from  Designing 
and  Conducting  Health  Survey  text  authored  by  Aday  and  Cornelius 


(2006) . 
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Step  1:  Develop  Survey  Tool 

For  this  policy  review,  the  criteria  for  Step  1,  Develop 
Survey  Tool,  includes,  an  assessment  of  each  option's  vision, 
evidence  based  content,  process,  and  length.  According  to  Aday 
and  Cornelius  (2006)  and  Drain  and  Clark  (2004),  identifying  the 
purpose  for  conducting  a  patient  satisfaction  survey  program  is 
essential  in  designing  an  effective  survey.  Aday  and  Cornelius 
assert  that  the  topic  of  interest  should  be  clearly  identified 
first  because  the  information  being  sought  should  determine  the 
appropriate  question  format. 

Establishing  the  vision  should  align  with  the  strategic 
intent  of  the  tool  and  should  determine  whether  a  survey  should 
evaluate  visit-specific  feedback,  or  evaluate  care  provided  over 
a  period  of  time.  To  receive  immediate  feedback  on  a  recent 
visit,  many  organizations  administer  a  short  questionnaire  on¬ 
site  following  an  appointment;  the  VA  Quick  Card  is  an  example 
of  this  type  of  evaluation  tool.  The  use  of  a  visit-specific 
survey  is  supported  by  literature  (Drain  &  Clark,  2004;  White 
1999).  Lengthy  and  complex  surveys  negatively  affect  response 
rates  and  data  quality.  Research  findings  conclude  that  low 
response  rates  diminish  the  survey's  internal  validity  (Aday  & 
Cornelius,  2006;  Drain  &  Clark,  2004;  White,  1999) . 

After  the  purpose  of  assessing  patient  satisfaction  is 
identified,  the  format  of  survey  items  should  be  determined. 
Ransom,  Joshi,  and  Nash  (2004)  recommended  that  the  length  of  a 
survey  item  be  limited  to  20  words,  but  Aday  and  Cornelius  (2006) 
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posit  that  longer  sentences  provide  the  patient  ample  time  to 
assess  the  information  in  support  of  a  more  precise  analysis. 
Andrews  (1984)  and  Bradburn,  Sudman,  and  Wans ink  (2004)  support 
the  conclusion  of  Aday  and  Cornelius,  noting  that  abbreviated 
questions  may  lead  to  carelessness.  Since  there  are  countering 
arguments  as  to  what  constitutes  the  appropriate  length,  the 
number  of  questions  were  assessed  in  order  to  determine  expected 
response  times  and  predict  CG  commander's  willingness  to  adopt 
the  policy.  Response  times  are  also  known  to  affect  response 
rates  and  will  be  discussed  later  in  this  section  (White,  1999)  . 

The  inclusion  of  sections  that  allow  verbatim  comments  was 
also  assessed,  as  supported  by  the  literature  (Press,  2006)  . 
Although  verbatim  questions  are  not  easy  to  tabulate,  their 
inclusion  constitutes  an  important  survey  element.  Open-ended 
responses  assist  in  understanding  conditions  behind  the  score 
(White,  1999 ) . 

Whether  the  policy  option  incorporated  evidence-based  survey 
methods  was  assessed.  Poorly  written  or  untested  questions, 
inadequate  design,  or  the  lack  of  psychometric  testing  may  lead 
to  inaccurate  responses  (Aday  &  Cornelius,  2006;  Jackson  & 

Kroenke,  1997)  .  Improvements  associated  with  surveys  that  lack 
validity  may  be  negligible  (Jackson  &  Kroenke,  1997;  Rubin, 

1997).  In  addition,  the  IOM  calls  for  the  increased  use  of 
evidence-based  practices  to  improve  the  efficiency  of  health 
care  (2001)  . 

Step  2:  Distribute  Survey 
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Distribute  Survey,  Step  2,  is  defined  by  the  criteria 
delivery  mode.  Handwritten,  mail,  telephone,  and  electronic 
surveys  are  examples  of  delivery  processes  that  have  been  used  to 
glean  patient  satisfaction.  A  reliable  and  effective  process  for 
assessing  patient  satisfaction  requires  standardizing 
distribution  because  the  methods  employed  may  affect  outcomes 
(Aday  &  Cornelius,  2006;  Drain  &  Clark,  2004;  Ransom,  Joshi  & 
Nash,  2006;  White,  1999).  Because  patients  sometimes  have 
difficulty  recalling  precise  details,  Ransom,  Joshi,  and  Nash 
(2006)  recommend  surveying  an  ambulatory  patient  immediately 
following  an  episode  of  care.  In  addition,  the  method  of 
distribution  should  be  consistent  (White,  1999)  .  This  policy 
review  reviewed  the  mode  of  delivery  of  each  policy  in  order  to 
determine  the  level  of  standardization. 

Step  3  -  Retrieve  Survey 

Step  3,  Retrieve  Survey,  is  closely  associated  to  Step  2. 

The  policy  review  of  survey  retrieval  assessed  the  application  of 
response  rates .  Response  rates  are  important  because  higher 
response  rates  support  a  survey's  validity  (Drain  &  Clark,  2004; 
Landon  &  Normand,  2006) .  Response  rates,  however,  are  difficult 
to  manage,  especially  among  small  practices.  Subsequently,  a 
policy  that  sets  standards  for  acceptability,  calculates  response 
rates,  and  garners  supported  from  clinic  staff  was  also 
evaluated . 

Step  4  -  Analyze  Data 

Once  the  data  has  been  retrieved.  Step  4,  Analyze  Data, 
occurs.  White  (1999)  posits  that  analyzing  data  may  be  the  most 
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complex  part  of  the  survey  process,  and  voices  concern  that  a 
lack  of  strong  analytic  skills  that  surveys  inhibits  data 
analysis.  Jackson  and  Kroenke  (1997),  note  that  results  of 
patient  satisfaction  analyses  are  often  stifled  by  internal 
evaluation,  due  to  the  potential  for  bias.  The  GAO  (1998)  also 
report  impartiality  concerns  over  internal  evaluations  of 
patient  satisfaction.  White  (1999)  states  that  outside  data 
analysis  generally  ranges  from  $300-$400  per  provider  for  small 
physician  practices;  therefore,  conducting  surveys  in-house 
could  be  more  cost  effective.  The  opportunity  cost  of 
accomplishing  the  analysis  in  house  as  well  as  the  cost 
investment  of  supporting  the  each  policy's  analysis  methodology 
was  assessed. 

In  addition  to  the  impact  of  impartiality  during  analysis, 
biases  of  respondents  should  also  be  assessed.  The  collection  of 
demographic  data  may  help  identify  how  certain  patients  respond 
to  particular  questions  and  determine  whether  subgroup 
differences  exist  (Aday  &  Cornelius,  2006;  White,  1999)  . 
Collection  of  demographic  data  and  sub-group  analysis  was  also 
evaluated . 

Step  5  -  Application 

Application,  Step  5,  evaluated  the  utility  of  outcomes. 
Griffith  and  White  (2002)  state  that  utility  exists  when  surveys 
are  "...sufficiently  detailed  to  identify  correctable 


43 


characteristics"  (p.  36).  Because  internal  benchmarking  may 
improve  replication  of  best  practices  and  advance  the 
implementation  of  change,  whether  the  policy  supported  facility 
metrics  and  local  improvement  initiatives  was  evaluated. 

A  comparable  score  from  a  validated  survey  tool  is  essential  in 
developing  and  reporting  results  to  external  stakeholders 
(Hibbard,  Stockard,  &  Tusler,  2003).  So  the  ability  to  externally 
benchmark  scores  was  also  evaluated. 

Policy  Analysis 

A  cost  analysis,  criteria  evaluation,  and  trade-off 
analysis  was  used  to  compare  the  three  policy  options.  The  cost 
analysis  was  split  into  two  segments,  implementation  costs  and 
operating  costs.  Implementation  costs  consisted  of  the  expected 
cost  associated  with  shifting  the  architecture  and  survey 
process  from  Option  1,  Status  Quo.  Operating  costs,  annual 
recurring  costs,  were  defined  as  the  expected  expense  of 
distributing  and  retrieving  the  survey,  as  well  as  the  expenses 
associated  with  analyzing  and  applying  survey  results. 

Evaluating  criteria  was  achieved  using  a  scorecard,  to 
include  a  matrix  of  the  desired  attributes,  as  presented  in  the 
Evaluative  Criteria  section  of  this  report  (p.  34-42).  The  cost 
analysis,  discussed  and  presented  first,  is  followed  by  the 
scorecard,  which  includes  a  summary  of  the  cost  analysis  and  the 
criteria  evaluation.  The  results  section  discusses  potential 
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trade-offs,  through  an  analysis  of  strengths  and  weaknesses  of 
each  policy  option.  Furthermore,  opportunity  costs  are 
acknowledged  and  considered.  Recommendations  and  policy 
selection  conclude  this  section. 

Cost  Analysis  Constructs 

The  cost  analysis  held  some  basic  constructs  constant  to 
ensure  the  policies  analyzed  were  comparative.  First,  the 
frequency  of  surveys  needed  to  be  established.  In  order  to 
define  Status  Quo,  Option  1,  13  of  the  42  CG  clinics  were 
contacted  and  the  frequency  survey  distribution  was  determined. 
Results  indicated  that  the  average  distribution  was  just  over 
twice  per  annum  (CG  Clinic  Administrators,  personal 
communication,  December,  2006) .  Thus,  a  semiannual  survey 
frequency  was  established  for  all  three  options. 

A  human  resources  cost  analysis  was  derived  using  a  common 
analysis  template  employed  by  resource  managers  at  DOD  MTFs  (LT 
Suzanne  Wood,  personal  communication,  September,  2007).  Subject 
matter  experts  at  TRICARE  South  Regional  Office  confirmed 
template  results  as  representative  of  costs  associated  with  CG 
manpower  expenditures  (Mr.  Dave  Montplaiser,  personal 
correspondence,  TRICARE  Business  Analyst,  September  2007) 
Subsequently,  a  fifty  dollar  hourly  resource  estimate  was 
determined  and  verified  by  CG  Headquarters  (LT  Timothy  Kulzer, 
personal  communication,  September  2007)  . 
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Implementation  Cost  Analysis 

No  implementation  costs  were  estimated  for  Option  1,  Status 
Quo,  because  under  this  policy  no  change  in  doctrine  was 
expected.  Operating  costs  and  opportunity  costs  are  associated 
with  Option  1  and  will  be  addressed  in  the  following  section. 

A  majority  of  the  costs  for  Option  2,  CG  Designed  Survey 
Web  Interface,  would  be  incurred  in  the  development  and 
implementation  phases  of  the  survey.  The  process  proposed  for 
designing  a  survey  and  developing  policy  included  the  assignment 
of  a  senior  headquarters  representative  and  a  project  manager. 

In  addition,  to  gain  buy-in  and  validate  survey  content,  a 
patient  satisfaction  conference  was  proposed.  Costs  for  the 
event  were  estimated  to  be  approximately  $30,000  and  would 
require  400  resource  hours  (See  Table  1) .  An  additional  480 
hours  was  projected  for  policy  implementation  and  survey 
testing.  A  total  of  2560  resource  hours  were  expected  to  be 
associated  with  the  conference,  development  and  implementation 
processes.  In  sum,  total  implementation  costs  were  projected  to 
be  $163,000  ([2,560*$50]  +  $30,000  =  $163,000).  Implementation 
cost  estimates  for  Option  2  were  validated  by  CG  Headquarters 
(LT  Timothy  Kulzer,  personal  communication,  September  2007). 

Option  3,  AMGA  Survey,  implementation  costs  were  estimated 
using  AMGA  Fee  Schedule  published  on  its  Website  (www.amga.org) . 
Estimates  included  costs  associated  with  policy  development  and 


46 

implementation.  CG  medical  command  was  expected  to  pilot  test 
the  AMGA  Survey  prior  to  implementing  the  policy  throughout  the 
organization.  The  AMGA  fee  was  estimated  at  $2,870  with  an 
additional  $5,000  for  100  resource  hours  allocated  to  pilot 
testing,  for  a  total  of  $7,870  (See  Table  1).  Policy  development 
and  dissemination  of  processes  was  projected  to  be  1,960 
resource  hours .  In  sum,  total  implementation  costs  were 
projected  to  be  $105,870  (  [2, 060*$50]  +  $2,870  =  $105,870). 
Implementation  cost  estimates  for  Option  3  were  validated  by  CG 
Headquarters  (LT  Timothy  Kulzer,  personal  communication, 

September  2007). 
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Table  1.  Design  and  Implementation  Costs 


Option  1 

Status  Quo 

Option  2 

CG  Designed  Web  Interface 

Option  3 

AMGA  Survey 

Design 

No  Cost  because 

Conference  Costs  =  $30,000 

Design  =  No  Cost 

and 

clinics  have  Conference  Resource  Costs  =  $20,000 

AMGA  Fee  =  $  2,870 

Test 

individual  surveys 

Test  Resource  Costs  =  $5,000 

Test  Resource  Costs  =  $  5,000 

Survey 

in  place 

Total  =  $55,000 

Total  =  $  7.870 

Policy 

No  Cost  because 

Policy  development  =  $  24,000 

Policy  development  =  $1 4,000 

Shift 

clinics  have 

Dissemination  =  $  84,000 

Dissemination  =  $84,000 

individual  surveys 
in  place 

Total  =  $108,000 

Total  =  $98,000 

Total 

$0 

$163,000 

$105,870 

Adopting  Option  2 

,  CG  Designed  Survey  Web 

Interface ,  or 

Option 

3,  AMGA  Survey, 

required  the  CG  to  invest  resources  to 

develop 

survey  methods 

and  processes.  Option  2, 

CG  Designed 

Survey 

Web  Interface, 

required  the  CG  to  design 

a  survey  in- 

house  and  was  estimated  at  $163,000.  Option  3,  AMGA  Survey,  at 
$105,870  was  projected  to  cost  a  third  less  than  Option  2.  The 
estimated  costs  of  implementing  new  policy,  presented  in  Table  1 
were  validated  by  CG  headquarters  command  (LT  Timothy  Kulzer, 
personal  communication,  September  2007).  This  $50,000  difference 
resulted  from  designing  a  survey  in-house  using  a  conference 
format . 

Two  divergent  possibilities  may  minimize  the  cost 
difference  between  Opt ion  2  and  Option  3.  First,  the  CG 
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customized  survey  could  be  designed  without  a  conference; 
however,  this  would  delay  policy  development  and  increase 
dissemination  costs.  Second,  the  possibility  exists  that  the  CG 
could  benchmark  against  a  standardized  survey  in  Web  format  and 
eliminate  the  design  aspect  associated  with  Option  2.  Adopting  a 
standard  survey  using  the  Design  Survey  tool  would  reduce 
estimated  costs,  thus  aligning  Option  2  with  Option  3,  totaling 
just  over  $100,000. 

Operating  Cost  Analysis 

Following  the  implementation  cost  analysis  the  annual 
recurring  costs,  or  operating  costs,  for  each  option  was 
evaluated.  Once  again,  the  $50  estimate  per  resource  hour  was 
used.  Correspondence  with  Clinic  Administrators  established  the 
basis  for  operating  estimates  (CG  Clinic  Administrators, 
personal  correspondence,  December  2006).  Option  1,  Status  Quo, 
served  as  benchmark  for  resource  hour  expenditures  per  activity: 

•  One  week  survey  period 

•  One  resource  hour  a  day  distributing  surveys 

•  One  resource  hour  a  day  retrieving  surveys 

•  16  resource  hours  per  survey  period  analyzing  results 

•  4  resource  hours  per  survey  applying  results. 

Option  1,  Status  Quo,  was  marked  by  variation  in  clinic 
processes  that  prohibited  systemic  comparisons  of  resource 
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consumption.  Estimates  for  Option  1  were  based  on  an  average 
survey  distribution  and  retrieval  cycle  for  42  clinics,  and  was 
estimated  at  ten  hours  per  survey,  840  hours  annually  (See  Table 
2).  Clinic-based  analyses  were  estimated  at  16  hours  per  survey 
period,  or  approximately  1344  hours  annually.  Application  of 
results  through  quality  improvement  efforts,  defined  as 
reporting  scores  to  the  patient  advocacy  committee,  was 
estimated  at  4  hours  per  survey,  or  336  hours  annually.  In  sum, 
total  annual  operating  costs  were  projected  to  be  $126,000 
(  [840*$50]  +  [1344*$50]  +  [1336*$50]  =  $126,000).  Annual 
operating  cost  estimates  for  Option  1  were  validated  by  CG 
Headquarters  (LT  Timothy  Kulzer,  personal  communication, 

September  2007) .  Within  Option  1,  Status  Quo,  there  were  no 
costs  associated  with  results  being  applied  strategically, 
because  the  current  process  lacked  standardization  and 
prohibited  cross-validation  of  metrics. 

In  determining  operating  costs  for  Option  2,  CG  Designed 
Survey  Web  Interface,  no  additional  fees  associated  with 
software  were  expected,  as  Design  Surveys  was  previously  funded. 
Collecting  and  retrieving  data  under  Option  2  would  reduce 
resource  hours  and  was  projected  to  be  5  hours  per  cycle,  or 
approximately  420  hours  annually  (See  Table  2) .  The  Web 
interface  included  an  automated  analysis  tool  that  would  reduce 
analysis  to  2  hours  per  cycle,  or  approximately  168  hours 
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annually.  Application  of  results  through  quality  improvement 
efforts,  defined  as  reporting  scores  to  the  patient  advocacy 
committee,  was  estimated  at  4  hours  per  survey,  or  336  hours 
annually.  However,  strategic  management  of  the  process  and 
application  of  the  results  required  an  increase  of  80  resource 
hours  in  external  benchmarking.  In  sum,  total  annual  operating 
costs  were  projected  to  be  $50,200  ( [420*$50]  +  [168*$50]  + 
[336*$50]  +  [80*$50]  =  $50,200).  Annual  operating  cost  estimates 
for  Option  2  were  validated  by  CG  Headquarters  (LT  Timothy 
Kulzer,  personal  communication,  September  2007). 

Operating  costs  for  Option  3  were  developed  by 
incorporating  the  cost  to  participate  in  the  AMGA  Survey  program 
with  the  cost  of  CG  resource  hours.  Recurring  external  fees  were 
calculated  for  250  providers  at  an  estimated  cost  of  $17,625 
annually  (See  Table  2).  Under  Option  3,  collecting  and 
retrieving  data  resource  hour  consumption  increased  and  was 
projected  to  be  15  hours  per  cycle,  or  approximately  630  hours 
annually.  This  increase  existed  because  AMGA  required  a  3 -week 
cycle  to  ensure  an  adequate  number  of  surveys  were  collected  for 
each  provider.  The  AMGA  Survey  program  included  analysis 
support;  however,  comment  section  analysis  would  remain  the 
responsibility  of  the  CG  clinic,  estimated  at  4  hours  per  cycle, 
or  approximately  336  hours  annually.  Similar  to  option  2,  there 
was  an  expected  need  for  increased  management  involvement. 
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Strategic  management  of  the  process  and  application  of  the 
results  required  an  increase  80  resource  hours  in  external 
benchmarking.  In  sum,  total  annual  operating  costs  for  Option  3, 
AMGA  Survey,  were  projected  to  be  $114,625  ($17,625  +  [630*$50] 

+  [336*$50]  +  [336*$50]  +  [80*$50]  =  $114,625).  Annual  operating 
cost  estimates  for  Option  3  were  validated  by  CG  Headquarters 
(LT  Timothy  Kulzer,  personal  communication,  September  2007). 


Table  2. 

Operating  Costs. 

Option  1 

Status  Quo 

Option  2 

CG  Designed  Survey 

Web  Interface 

Option  3 

AMGA  Survey 

Distribute 

and 

Retrieve 

Survey 

Resource  Costs  = 
$42,000 

Resource  Costs  = 
$21,000 

AMGA  Membership  Fee  and 
Processing  Fees  for  250 
providers  =  $17,625 
Resource  Costs  =  $63,000 
Total  =  $80,625 

Analyze 

Survey 

Resource  Costs  = 
$67,200 

Resource  Costs  = 
$8,400 

Resource  Costs  = 
$16,800 

Clinic 

Application 
of  Survey 
Results 

Resource  Costs  = 
$16,800 

Resource  Costs  = 
$16,800 

Resource  Costs  = 
$16,800 

Coast 

Guard 

Wide 

Application 
of  Survey 
Results 

No  Cost,  because  it  can 
not  be  done  under  current 
policy. 

Resource  Costs  = 
$4000 

Resource  Costs  = 
$4000 

Total 

$126,000 

$50,200 

$114,625 
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The  review  of  annual  operating  expenses  showed  that  the 
most  economically  feasible  alternative  was  Option  2,  CG  Designed 
Survey  Web  Interface,  estimated  at  just  over  $50,000  a  year. 
Differences  in  option  costs  resulted  from  the  implementation  of 
technology.  Specifically,  the  use  of  a  Web  interface,  which 
minimizes  distribution  and  retrieval  while  automating  analysis, 
reduced  resource  hour  investments  associated  with  Option  2. 
Additional  savings  associated  with  Option  2  were  anticipated 
because  the  policy  proposes  to  use  a  software  program,  Design 
Surveys,  which  is  already  available  to  the  CG  clinics. 

Option  3,  AMGA  Survey  ($114,625),  and  Option  1,  Status  Quo 
($126,000),  were  estimated  to  be  comparable.  However,  Option  3, 
AMGA  Survey,  required  annual  fees  estimated  at  $18,000. 
Additional  budget  allocations  may  be  questioned,  as  current 
patient  satisfaction  evaluation  processes  were  devoid  of 
external  expenditures. 

Criteria  Analysis 

The  Criteria  Analysis  is  presented  in  a  scorecard  that 
includes  a  matrix  with  options  presented  in  relation  to  desired 
attributes  (see  Table  3).  Included  in  the  scorecard  are  the 
estimated  implementation  costs  per  cycle  (from  Table  1)  as  well 
as  a  project  cost  per  clinic  survey  derived  from  Operating  Costs 
(see  Table  2) .  The  desired  criteria  used  in  the  scorecard  was 
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presented  graphically  in  Figure  3,  p.  37,  and  discussed  in  depth 
in  the  Evaluative  Criteria  section  (p  34.  -  p  42 . ) 

Option  1,  Status  Quo,  was  defined  by  evaluating  clinic 
surveys  and  questioning  clinic  administrators;  13  of  the  42 
clinics  were  evaluated.  The  mean  number  of  questions  for  CG 
surveys  sampled  was  17.2,  with  6  questions  being  shortest  and  40 
questions  being  the  longest  (Table  3).  Most  clinics  included  at 
least  one  verbatim  response  question  on  their  surveys. 

A  review  of  the  questions  from  all  13  clinics  found  that  4 
clinics  did  not  request  feedback  on  overall  patient 
satisfaction.  The  remaining  nine  clinics  assessed  overall 
patient  satisfaction  in  four  distinct  ways: 

•  Overall  rate  visit 

•  Overall  rate  clinic 

•  Would  you  recommend  provider 

•  Would  you  recommend  facility 

Administrators  who  favored  the  use  of  patient  satisfaction 
surveys  directly  linked  change  initiatives  to  verbatim  comments 
related  by  topic,  not  patient  satisfaction  scores.  The  clinic 
administrators  interviewed  were  asked  how  many  times  a  year 
their  clinics  surveyed  patients.  A  majority,  8  of  the  13  clinics 
exceeded  the  annual  requirement  to  survey.  Clinics  that  surveyed 
more  frequently  utilized  processing  tools  like  Excel  or  Design 
Surveys  to  automate  analyses . 
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Option  1,  Status  Quo ,  empowered  each  clinic  to  implement 
there  own  survey  process,  preventing  internal  as  well  as 
external  benchmarking.  Only  1  of  the  13  clinics  implemented  an 
evidence-based  tool.  Also,  current  processes  rarely  evaluated 
response  rates  or  analyzed  demographic  information;  thus,  the 
validity  of  the  current  patient  satisfaction  scoring  program  for 
CG  was  unreliable.  With  no  ability  to  discern  an  organizational 
patient  satisfaction  score,  the  question  remained:  At  an 
estimated  annual  operating  cost  of  $1500  per  survey  per  clinic, 
does  the  cost  of  the  current  CG  patient  satisfaction  program 
outweigh  the  benefit? 

Option  2,  CG  Designed  Survey  Web  Interface,  supported 
internal  benchmarking  and  organizational  measures,  while 
providing  the  CG  the  flexibility  in  determining  the  form  and 
content  of  an  organizational  survey  (see  Table  3) .  The  low  cost 
of  sending  surveys  and  survey  reminders  was  a  technology  plus . 
Additionally,  analyses  were  automated  and  errors  in  uploading 
and  analyzing  data  were  reduced. 

However,  there  were  a  number  of  detractors  for  Option  2,  CG 
Designed  Survey  Web  Interface .  First,  the  survey  would  not  be 
valid  due  to  a  lack  of  psychometric  testing.  Second,  an 
organization-specific  design  prohibited  external  benchmarking. 
The  third  concern  was  with  the  distribution  method  of  the 
survey,  as  only  a  small  percentage  of  clinics  in  the  CG,  2  of 
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13,  queried  patients  by  sending  electronic  surveys  to  patients. 
With  the  pencil-paper  method  overwhelmingly  implemented  in  the 
CG,  an  electronic  survey  method  may  affect  survey  participation, 
quality  of  responses,  and  validity  of  scores  (OMB,  2006) .  In 
2006  the  Office  of  Management  and  Budget  (OMB)  reported  that 
paper  surveys  typically  achieve  higher  response  rates  than  web 
surveys  or  email  surveys.  OMB  also  posited  that  low  response 
rates  may  result  from  respondents'  concerns  over 
confidentiality. 

Electronic  surveys  have  not  achieved  broad-based 
implementation  because  a  perception  exists  that  distributing 
surveys  electronically  could  prevent  a  certain  segment  of 
population  from  responding,  biasing  results  (Aday  &  Cornelius, 
2006;  OMB,  2006).  This  is  not  a  perceived  detractor  of 
introducing  an  electronic  survey  to  evaluate  patient 
satisfaction  in  the  CG,  as  a  majority  of  respondents  are  CG 
active  duty  members;  all  CG  members  are  provided  access  to 
computers  and  are  considered  proficient  in  basic  computer  use. 

Option  3,  AMGA  Survey,  queried  patients  using  a  pencil- 
paper  survey  method,  similar  to  current  practices,  Status  Quo; 
however,  it  employed  an  evidence-based  tool  whose  results  were 
calculated  externally  to  the  organization  (see  Table  3).  Using  a 
tested,  validated,  standardized  tool  that  supported  internal  and 
external  benchmarking  of  clinics,  as  well  as  providers, 
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increased  the  utility  and  application  of  results.  The  AMGA 
Survey  program  allowed  an  organization  to  customize  3  questions, 
affording  the  CG  limited  flexibility  in  aligning  the  survey  to 
particular  CG  wants  and  needs. 

Remaining  with  a  pencil -paper  approach  did  not  embrace 
technology,  and  detractors  may  perceive  it  as  rudimentary  in 
design.  Other  concerns  included  AMGA  policy  which  encouraged 
survey  distribution  over  a  three-week  period,  which  is  longer 
than  current  practice  (one  week) .  As  results  were  aggregated  by 
provider,  some  providers  may  balk  at  additional  oversight. 
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Table  3.  Policy  Analysis  Scorecard. 


Option  1 

Option  2 

Option  3 

CG  Designed  Web 

Status  Quo 

Interface 

AMGA  Survey 

Implementation 

Costs 

None 

$163,000 

$105,870 

Clinic  Operating 
Costs  Per  Cycle 

$1,500 

$598 

$1,365 

Number  of 

Questions 

6  to  40 

15  to  25 

15  to  18 

Ability  to 

Change  Questions 

Yes 

Yes 

No* 

Evidence  Based 
Metrics 

No 

No 

Yes 

Survey  Delivery 

Mode 

In  person 

Web 

In  person 

Computed 

Response  Rates 

No 

No 

Yes 

Automated 
External 
Analysis  of 
Responses 


Yes 


Yes 


Demographics 


No 


TBD 


Yes 


Benchmark 

Internally  No  Yes  Yes 


Benchmark 

Externally  No  No  Yes 


Strategic  Intent 

Metrics  No  Yes  Yes 


*  AMGA  allowed  3  questions  to  be  customized. 


**  A  few  of  the  CG  Clinic  surveys  were  Web-based  incorporating  external 
analysis;  however,  a  majority  of  CG  clinic  required  CG  administrator  to 
analyze  results  internally. 
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Recommendations 

The  current  method,  Option  1,  Status  Quo,  at  a  cost  $1500 
is  not  evidence-based  and  results  remain  limited  in  application. 
Opt ion  2,  CG  Designed  Survey  Web  Interface,  incurs  substantial 
implementation  costs,  and  results  prevent  external  application 
of  metrics.  Option  3,  AMGA  Survey,  is  the  best  policy.  Using  the 
AMGA  Survey  program  provides  the  CG  a  uniform,  evidence-based 
tool  that  is  distributed  in  accordance  with  current  practices. 
Also,  external  examination  reduces  internal  analyses  and 
improves  validity  by  decreasing  bias.  Adopting  Option  3  would 
provide  survey  results  that  allow  for  internal  and  external 
benchmarking,  delineates  provider-specific  metrics,  as  well  as 
provides  a  foundation  for  organizational  metrics.  At  a  cost  of 
$1,365  per  survey  per  clinic,  the  AMGA  costs  less  than  the 
current  process  (Option  1) .  Although  the  operating  cost  estimate 
for  Option  3,  AMGA  Survey  ($1,365),  is  more  than  Option  2,  CG 
Designed  Survey  Web  Interface  ($598),  the  intangible  benefits 
outweigh  expenditures. 

Conclusions 

Mission  requirements  of  the  CG  have  increased  the  demand 
for  health  care.  Literature  supports  the  evaluation  of  patient 
satisfaction  as  a  well-documented  impetus  for  initiating  measures 
to  improve  the  quality  of  health  care  (Aday  &  Cornelius,  2006; 
Griffith  &  White,  2002;  Press,  2006;  Ransom,  Joshi,  &  Nash, 
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2005) .  The  CG,  however,  lacks  a  quality,  evidenced-based  system, 
for  measuring  how  demands  and  service  changes  affect  patient 
perceptions . 

This  study  provides  an  analysis  of  how  the  CG  currently 
evaluates  patient  satisfaction  throughout  the  organization. 
Findings  determined  how  the  CG  evaluates  patient  satisfaction  is 
inefficient  and  ineffective,  as  many  opportunities  exist  to 
improve  the  evaluation  processes.  Unfortunately,  an  ineffective 
patient  satisfaction  system  limits  the  quality  of  patient  care 
(IOM,  2006;  Press,  2006),  thus  a  more  effective  system  for 
assessing  patient  satisfaction  is  needed. 

The  use  of  a  standard  rating  system  provides  structure 
and,  in  accordance  with  Donabedien  (1998),  structure  is  the 
foundation  for  improving  the  quality  of  care.  This  report 
concludes  that  standardization  within  the  CG  evaluation  system 
should  be  a  priority,  as  standard  structure  supports  quality 
outcomes.  Quality  outcomes  are  the  foundation  for  benchmarking; 
benchmarking  is  a  management  tool  that  increases  the 
institutions  ability  to  adapt  to  rapid  change  and  adopt 
improvements,  thus  supports  a  continuous  improvement  environment 
(Bogan  &  English,  1994)  .  In  addition,  adopting  a  standard  survey 
that  assesses  patient  satisfaction  similarly  in  all  CG  clinics, 
allows  for  system-wide  evaluations,  laying  the  groundwork  for 
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metrics  that  could  be  used  to  measure  effectiveness,  leverage 
resources,  and  increase  financial  support. 

Study  results  pertain  to  patient  satisfaction  evaluations 
conducted  in  CG  clinics;  however,  a  significant  percentage  of  CG 
patient  care  occurs  externally.  A  visit-specific  survey 
standardized  and  distributed  evenly  at  CG  clinics  fulfills  only 
some  of  the  needs  of  the  CG  health  care  system.  Thus, 
considering  the  complexity  of  the  CG's  care  model,  this  report's 
findings  are  limited.  The  CG  must  improve  the  clinic  evaluation 
process  and  advance  the  monitoring  of  patient  perceptions  for 
those  seeking  care  externally.  A  systems  approach,  to  include 
internal  and  external  care  evaluations,  is  ideal. 

In  adopting  policy  to  standardize  patient  satisfaction 
evaluations,  the  CG  must  review  the  application  of  results. 
Determining  how  the  scores  will  be  used  in  both  operational  and 
strategic  settings  should  be  identified  and  conveyed  to 
stakeholders.  Additionally,  communicating  how  survey  results 
will  be  applied  to  clinic  practice  and  budget  planning  must  also 
be  addressed. 

Furthermore,  adopting  a  new  survey  must  be  accompanied  by 
quality  assurance  efforts,  as  in  addressing  maintenance  of  the 
patient  satisfaction  feedback  process.  Awareness  of  competitive 
trends  in  surveying  patient  perceptions  should  be  sought.  This 
recommendation  is  reinforced  by  actions  of  the  TRICARE 
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Management  Activity  (TMA) ,  the  governmental  office  that  overseas 
MHS  operations.  TMA  chartered  a  working  group  in  2006  to 
facilitate  the  efficient  and  effective  development  of  MHS 
surveys .  The  CG  should  draw  on  this  resource  in  developing  and 
maintaining  current  evaluation  practices . 

Accompanying  support  within  the  organization  is 
acknowledged.  Admiral  Allen  (2007),  the  Commandant  of  the  CG, 
issued  ten  "Commandant  Intent  Action  Orders"  to  focus  strategic 
efforts.  Standardizing  CG  patient  satisfaction  system  aligns 
with  several  of  the  action  orders  including 

•  Realign  the  Coast  Guard's  logistics  organization  to 
reduce  the  burden  on  our  field  units,  control  costs 
and  improve  accountability. 

•  Develop  a  new  operational  framework  for  the  Coast 
Guard  to  execute  our  missions  more  effectively  and 
efficiently,  (p.2-3) 

In  summary,  patient  satisfaction  evaluation  is  just 
one  of  many  management  tools  that  can  be  standardized  within  CG 
health  care.  Since  high  patient  satisfaction  is  a  predictor  of 
high  quality  care,  improving  this  feedback  mechanism  could  be  an 
impetus  for  affecting  decision  making  constructs  throughout  the 
CG  health  care  system. 
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